Identification method using partitioning of subject silhouettes

ABSTRACT

Described herein is an apparatus and method for gait recognition. The apparatus includes circuitry that is configured to receive a gait sequence including a predetermined number of image frames of a subject. The received gait sequence is processed to generate a gait-energy-image (GEI). A plurality of Gabor filter responses is computed by convoluting the generated GEI with a bank of Gabor filters, wherein the filter bank includes a first predetermined number of unique scales, and a second predetermined number of unique orientations. Further, the circuitry is configured to partition, each Gabor filter response of the computed plurality of Gabor filter responses, into a predetermined number of overlapping regions and extract, a predetermined number of statistical features only from the overlapping regions, the extracted statistical features corresponding to texture content of the subject. The circuitry eventually recognizes the subject based on a classification of the extracted statistical features.

INCORPORATION BY REFERENCE

The present disclosure claims the benefit of U.S. ProvisionalApplication No. 62/276,563, filed on Jan. 8, 2016, which is incorporatedherein by reference in its entirety.

FIELD OF INVENTION

The present disclosure relates generally to a technique of gaitrecognition based on an analysis of overlapping Gabor-based regions.

BACKGROUND

The background description provided herein is for the purpose ofgenerally presenting the context of the disclosure. Work of thepresently named inventors, to the extent the work is described in thisbackground section, as well as aspects of the description that may nototherwise qualify as prior art at the time of filing, are neitherexpressly nor impliedly admitted as prior art against the presentdisclosure.

Gait recognition is becoming an increasingly popular research problemfor human identification based on a walking style of the subject. Gaitrecognition has emerged as an attractive research problem as itpossesses several desirable merits unlike other biometrics. However,most of the existing gait recognition methods (including those thatinvolve Gabor-based filters) suffer from the curse of dimensionality.Some techniques employ a dimensionality reduction process, in order tomake the gait recognition process feasible. However, such processesstill incur a high computational and storage cost, and further incurdifficulties in identifying subjects with a high degree of confidence.

Biometric recognition has been widely used as a powerful tool forautomatic human identification and authentication. The gait biometricidentifies subjects (i.e., people) by their way of walking. Unlike otherbiometrics like face and fingerprint, gait recognition does not requestthe target subject to interact in a predefined and cooperative mannersuch as being close to the acquisition device or standing at a specificangle. In gait-based systems, the process of image acquisition isnon-intrusive. Thus, it can be done in public areas without attractingthe attention of subjects under surveillance. Also the system can workat longer distances (e.g. 10 m or more), unlike most of the otherbiometrics. Moreover, the gait modality is difficult to disguise and canbe of low resolution.

Gait recognition processes however have certain limitations. The processcan be greatly affected by a number of conditions like type of shoes,clothes, and the like that are worn by the subject in effectivelyrecognizing the subject. Furthermore, the discriminating power ofwalking style can also be degraded by certain physical factors such asinjuries. Nevertheless, as described by I. Bouchrika et al. in, “Towardsautomated visual surveillance using gait for identity recognition andtracking across multiple non-intersecting cameras,” Multimedia Tools andApplications, 2014, which is incorporated herein by reference in itsentirety, gait is still a potential choice for intelligent visualsurveillance and tracking of subjects.

Gait recognition processes can be classified broadly in two maincategories: model based methods and model-free methods. In model basedmethods such as those described by D. K. Wagg et al. in, “On automatedmodel-based extraction and analysis of gait,” proceedings of the 6thIEEE International Conference on Automatic Face and Gesture Recognition,2004, pp. 11-16; C. Yam et al. in, “Automated person recognition bywalking and running via model-based approaches,” Pattern Recognition,vol. 37, no. 5, pp. 1057-1072, 2004; and by L. Wang et al. in,“Automatic gait recognition based on statistical shape analysis,” IEEETransactions on Image Processing, vol. 12, no. 9, pp. 1120-1131, 2003,which are incorporated herein by reference in their entirety, the searchfor the human movement parameters is usually guided by a statistical orgeneric model. Here, the frequency and amplitude are typically mergedwith extracted features, or the collection of images is directly used.

In contrast, the model-free approaches such as those described by J. E.Boyd et al. in “Phase in model-free perception of gait,” in HumanMotion, 2000, proceedings workshop on, 2000, pp. 3-10, and Y. Dupuis etal. in “Feature subset selection applied to model-free gaitrecognition,” Image and vision computing, vol. 31, no. 8, pp. 580-591,2013, which are incorporated herein by reference in their entirety, usestatic and dynamic components. The static components reflect the shapeand size of a human body, whereas the dynamic components reflect themovement dynamics. Examples of static features are height, width, stridelength, and silhouette bounding box lengths, whereas frequency and phaseof movement are examples of dynamic features. Furthermore, model-freetechniques may also be classified into temporal and spatio-temporalmethods. The research on model-free systems is relatively more than thaton model-based systems, because of the computationally tractability.

Temporal gait recognition approaches are expensive in terms of storageand computation due to the frame by frame feature extraction andclassification. Gait energy images (GEIs) represent the human walking ina single image conserving motion temporal properties. Several gaitrecognition approaches rely on features extracted from GEIs. However,such gait recognition processes use reduced-dimensionality GEIs, orapply the feature extraction algorithm on the holistic GEI.

Additionally, Gabor filters have been widely used as an effectivefeature extraction approach in many fields of research. Such filtershave been also utilized in many biometric applications such as irisrecognition, and face recognition. However, a main problem incurred withthe use of the Gabor filter is the huge dimensionality caused by aconvolution process. Few attempts have been proposed for using Gaborfilters in gait recognition. For example, the work of Huang et al. “Gaitrecognition based on Gabor wavelets and modified gait energy image forhuman identification,” Journal of Electronic Imaging, vol. 22, no. 4,October 2013, which is incorporated herein by reference in its entirety,applied a Gabor filter on a modified version of GEI representation. Athree-step search (TSS) algorithm was used to prevent the confliction oftreating multi-walkers in a silhouette as one person. Adaptivebackground model was also utilized to alleviate the effect ofillumination variance and slow walking. The dimensionality was reducedusing PCA and an SVM with RBF-kernel was trained and used forclassification.

Accordingly, the present disclosure provides for a framework of gaitrecognition and characterization, which is not computationally and/orstorage wise intensive. The gait recognition technique of the presentdisclosure provides an efficient and cost-effective manner ofrecognizing objects.

SUMMARY

An aspect of the present disclosure provides a technique of gaitrecognition. Specifically, a statistical gait recognition approach isdescribed herein that is based on the analysis of overlappingGabor-based regions. The Gait Energy Image (GEI) is first constructedfrom the gait sequence as a spatio-temporal summary. Further, by oneembodiment of the present disclosure, the GEI image is convolved with aGabor filter bank of 8 different orientations and 5 different scales. Astatistical analysis is thereafter applied to extract discriminativegait features from multi-overlapped Gabor-based regions. Consecutively,an SVM classifier is utilized to measure the gait similarity andidentify the subject.

An aspect of the present disclosure provides for a method of gaitrecognition, the method comprising: receiving by circuitry a gaitsequence, the gait sequence including a predetermined number of imageframe of a subject; processing by circuitry, the received gait sequenceto generate a gait-energy-image (GEI); computing by circuitry, aplurality of Gabor filter responses by convoluting the generated GEIwith a bank of Gabor filters, the filter bank including a firstpredetermined number of unique scales, and a second predetermined numberof unique orientations, partitioning by circuitry, each Gabor filterresponse of the computed plurality of Gabor filter responses, into apredetermined number of overlapping regions; extracting by circuitry, apredetermined number of statistical features only from the overlappingregions, the extracted statistical features corresponding to texturecontent of the subject; and recognizing by circuitry, the subject basedon a classification of the extracted statistical features.

Another aspect of the present disclosure provides for a non-transitorycomputer readable medium having stored thereon a program that whenexecuted by a computer causes the computer to execute a method of gaitrecognition, the method comprising: receiving a gait sequence, the gaitsequence including a predetermined number of image frames of a subject;processing the received gait sequence to generate a gait-energy-image(GEI); computing a plurality of Gabor filter responses by convolutingthe generated GEI with a bank of Gabor filters, the filter bankincluding a first predetermined number of unique scales, and a secondpredetermined number of unique orientations, partitioning by circuitry,each Gabor filter response of the computed plurality of Gabor filterresponses, into a predetermined number of overlapping regions;extracting a predetermined number of statistical features only from theoverlapping regions, the extracted statistical features corresponding totexture content of the subject; and recognizing the subject based on aclassification of the extracted statistical features.

According to another aspect of the present disclosure is provided anapparatus comprising: circuitry configured to receive a gait sequence,the gait sequence including a predetermined number of image frames of asubject; process the received gait sequence to generate agait-energy-image (GEI); compute a plurality of Gabor filter responsesby convoluting the generated GEI with a bank of Gabor filters, thefilter bank including a first predetermined number of unique scales, anda second predetermined number of unique orientations, partition eachGabor filter response of the computed plurality of Gabor filterresponses, into a predetermined number of overlapping regions; extract apredetermined number of statistical features only from the overlappingregions, the extracted statistical features corresponding to texturecontent of the subject; and recognize the subject based on aclassification of the extracted statistical features.

The foregoing paragraphs have been provided by way of generalintroduction, and are not intended to limit the scope of the followingclaims. The described embodiments together, with further advantages,will be best understood by reference to the following detaileddescription taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of this disclosure that are provided as exampleswill be described in detail with reference to the following figures,wherein like numerals reference like elements, and wherein:

FIG. 1A depicts an exemplary flowchart illustrating the steps performedin gait recognition by one embodiment;

FIG. 1B depicts exemplary boundaries achieved by a segmentation process;

FIG. 2 depicts an exemplary Gabor patterns including overlappingregions; and

FIG. 3 illustrates an exemplary block diagram of a computing device.

DETAILED DESCRIPTION OF EMBODIMENTS

Exemplary embodiments are illustrated in the referenced figures of thedrawings. It is intended that the embodiments and figures disclosedherein are to be considered illustrative rather than restrictive. Nolimitation on the scope of the technology and of the claims that followis to be imputed to the examples shown in the drawings and discussedherein.

By one embodiment of the present disclosure is provided a methodologyfor gait recognition. The method is based on the generation of GEI andGabor filters. However, unlike earlier Gabor-based approaches for gaitrecognition, which mainly utilize the whole Gabor responses, the methodaccording to an embodiment, applies statistical feature extraction onlyin overlapping regions.

turning to FIG. 1A is depicted an exemplary flowchart 100 depicting thesteps performed in gait recognition by one embodiment of the presentdisclosure.

A sequences of images (i.e. a gait sequence) 101 is preprocessed 103 togenerate a GEI image. For instance, by one embodiment. Gait Energy Image(GEI) is a sum of images of a walking silhouette dived by the number ofimages. GEI is a useful representation with superior selective power andstrength against segmental errors. The GEI image can be computed bypre-processing a binary gait silhouette image B_(t)(x, y) at time t in asequence GEI is computed as:

${{{GEI}( {x,y} )} = {\frac{1}{N}{\sum\limits_{t - 1}^{N}\; {{Bt}( {x,y} )}}}},$

where N is the number of frames in the full gait cycle and x and y arespatial coordinates of pixels in the image.

The generated GEI image (from the output of 103) is convolved with abank of Gabor filters 105. Specifically, circuitry (described later withreference to FIG. 3) that is included in an image processing device maybe configured to perform a convolution operation of the generated GEIimage and the Gabor bank of filters.

By one embodiment, the GEI gait image is represented as a functionGEI(x,y), which is convolved with a bank of Gabor filters (i.e., aconvolution operation of the GEI gait image is performed with respect tothe Gabor filters). The filter bank has 5 different scales and 8different orientations. The result of the convolution process can beexpressed as:

G _(ν,μ)(x,y)=GEI(x,y)*ψ_(ν,μ)(x,y)  (1)

where * represents convolution, ψ_(ν,μ)(x,y) is a 2D Gabor waveletkernel function at orientation μ and scale ν, and G_(ν,μ)(x,y)represents the convolution output. The kernel is defined as:

$\begin{matrix}{{\psi_{v,\mu}(z)} = {\frac{{k_{v,\mu}}^{2}}{\sigma^{2}}{e^{- {({{k_{v,\mu}}^{2}{{z}^{2}/2}\sigma^{2}})}}\lbrack {e^{i\; k_{v,\mu}z} - e^{{- \sigma^{2}}/2}} \rbrack}}} & (2)\end{matrix}$

z=(x,y), ∥●∥ is the Euclidean norm operator, k_(ν,μ)=k_(ν)e^(iφμ) withk_(ν)=k_(max)/λ^(ν), λ is the spacing factor between FIG. 1A Gaborwavelets in the frequency domain, and φ_(μ) is the orientation. TheGabor response includes two portions: a real part R_(ν,μ)(x,y), and animaginary part Im_(ν,μ)(x,y). By one embodiment, the magnitude of theGabor response is used in gait recognition. The magnitude can becomputed as follows:

Mag _(ν,μ)(x,y)=√{square root over (R _(ν,μ) ²(x,y)+Im _(ν,μ)²(x,y))}  (3)

Further, the convolution operation in 105 results in a plurality ofGabor responses. Each response of the plurality of Gabor responses ispartitioned into a predefined number of unique (i.e., different sized)overlapping regions. For instance, by one embodiment, in order tocapture as much discriminative gait information as possible, theobtained 40 Gabor responses (corresponding to a Gabor filter including 5different scales and 8 different orientations) are partitioned intopredefined different-sized overlapping regions. By one embodiment,different partitioning scenarios may be attempted to test whichpartitioning is more effective to gait applications. An exemplaryoverlapping of regions of Gabor patters is depicted in FIG. 2.Specifically, FIG. 2 depicts an example of a Gabor response partitioning(201, 203, 205, and 207) of 5 main overlapping body regions. Eachcolored line represents a boundary of a region. The length of the commonarea between each two adjacent regions can be set to a predeterminednumber of pixels such as 10 pixels.

Turning back to FIG. 1A, upon segmentation of the Gabor responses beingcomplete, a region-level statistical analysis is performed only on theoverlapping regions of the Gabor responses in 109.

By one embodiment, partitioning is conducted as depicted in FIG. 1B.Specifically, FIG. 1B depicts the partitioning 150 of a sequence ofimages. The portioning is performed as a fraction of the subject's 155height and width, and is denoted by horizontal and vertical lines, 160and 165 respectively. The underlying idea is to separate moving partssuch as head, arms, legs, etc. After normalization and alignment of GEI,the boundaries between regions are set. For instance, the head part isset to include about 19% of the whole subject's height, and 10 pixelsare set as the length of the common area between each two adjacentregions.

By one embodiment, in the Gabor based gait feature extraction process,the features of mean (average) and a standard deviation are extractedfrom the overlapping regions only, to describe the gait. However, thesetwo statistical features may not be enough, and may lead to the loss ofimportant gait information in the Gabor patterns. Consequently, by oneembodiment, a total of 6 more statistical features are utilized tocapture the texture content which participates in notable improvement ofthe overall gait recognition performance.

Specifically, statistical features are extracted from each overlappingregion of each Gabor pattern. Further, the obtained set of features ismerged to form a global feature vector that is to be used in trainingand classification. By one embodiment, using a normalized co-occurrencematrix, represent as p_(i,j), the utilized statistical features are asfollows:

Energy: measures the uniformity of an image as follows:

$\begin{matrix}{E = {\sum\limits_{i,j}\; p_{i,j}^{2}}} & (4)\end{matrix}$

Correlation: measures to what level a pixel in an image are correlatedto its neighbors; it is given by:

$\begin{matrix}{{Cor} = {\sum\limits_{i,j}\; \frac{( {i - \mu_{i}} )( {j - \mu_{j}} )p_{i,j}}{\sigma_{i}\sigma_{j}}}} & (5)\end{matrix}$

Contrast: measures the intensity local variation in the co-occurrencematrix:

$\begin{matrix}{{Con} = {\sum\limits_{i,j}\; {( {i - j} )p_{i,j}}}} & (6)\end{matrix}$

Entropy: measures randomness of disorder of an image. Highest value isobtained when all intensities are the same. It is given by:

$\begin{matrix}{{Ent} = {\sum\limits_{i,j}\; {p_{i,j}\log_{2}p_{i,j}}}} & (7)\end{matrix}$

Homogeneity: measures the spatial closeness of the distribution ofelements in an image:

$\begin{matrix}{{Hom} = {\sum\limits_{i,j}\; \frac{p_{i,j}}{1 + {{i - j}}}}} & (8)\end{matrix}$

Smoothness: measures the number of times the density function can bedifferentiated:

$\begin{matrix}{{Smo} = {1 - \frac{1}{\sigma^{2}}}} & (9)\end{matrix}$

Furthermore, by one embodiment, in the gait classification step of FIG.1A, a support vector machine (SVM) classifier with a linear kernel isused for gait recognition using the extracted feature vectors,whereafter the process as depicted in FIG. 1A terminates.

By one embodiment, SVMs are a set of supervised learning methods usedfor classification, regression and outlier detection. Utilizing SVMsprovides the advantageous ability of being effective in high dimensionalspaces, effective in cases where number of dimensions is greater thanthe number of samples, using a subset of training points in the decisionfunction (called support vectors), so it is also memory efficient, andbeing versatile, i.e., different kernel functions can be specified forthe decision function.

By one embodiment, the gait recognition method as described withreference to FIG. 1A, is evaluated on the CASIA B gait database asdescribed by S. Yu et al. in, “A framework for evaluating the effect ofview angle, clothing and carrying condition on gait recognition,” inProceedings of the 18th International Conference on Pattern Recognition(ICPR), vol. 4, 2006, pp. 441-444, and incorporated herein by referencein its entirety. The database includes sequence samples of 124 subjectsincluding 93 males and 31 females.

By one embodiment, gait sequences for each subject are captured from 11different views. Each subject walks 10 times through a straight line ofconcrete ground (6 normal walking, 2 wearing a coat, and 2 carrying abag). At each walking instance, there are 11 cameras capturing thesubject walking. Consequently, each subject has a total of 110 videosequences and the database includes 110×124=13640 total sequences forall subjects.

By one embodiment, the CASIA B database is divided into three rest sets:normal walking, carrying a bag, and wearing a coat. The division of thedatabase provisions for testing the gait recognition approach of thepresent disclosure to identify subjects under different scenarios.Firstly, the gait recognition method of the present disclosure (referredto herein after as ‘GR method’ is applied on the Gabor patterns withoutany partitioning. Tables I, II, and III depict the recognition rates ofthe GR method and illustrate a comparison of the GR method to otherimplemented methods for the three scenarios. It must be appreciated thatall comparisons are conducted in terms of recognition rate. The resultsas shown in Tables I, II, and III demonstrate that the statisticalapproach of the GR method outperforms all other methods under allscenarios. Moreover, the performance of the GR method is superior in thecase of probe of subjects wearing coat.

TABLE I Comparison of correct classification rates (CCR) for NormalWalking scenario of CASIA B without partitioning Mean + GR- Angle (φ)pHOG LXP LBP SLBP std Method 0 82.76 61.64 56.9 68.54 88.79 95.26 1874.57 61.21 66.81 65.52 86.64 97.41 36 76.72 53.02 60.35 61.21 76.2991.38 54 76.72 56.04 56.9 63.79 79.31 92.24 72 81.47 60.78 68.54 68.5484.91 91.81 90 86.21 62.07 73.28 68.97 80.6 93.1 108 81.04 63.36 68.9765.52 83.19 93.97 126 77.59 57.33 62.5 68.54 81.9 91.81 144 76.72 57.3361.21 66.81 79.74 91.38 162 78.45 63.79 68.97 75.43 83.19 92.24 18083.62 53.02 57.33 66.38 87.07 95.26

TABLE II Comparison of correct classification rates (CCR) for CarryingBag scenario of CASIA B without partitioning. Mean + GR- Angle (φ) pHOGLXP LBP SLBP std Method 0 45.26 26.72 28.02 28.45 45 64.66 18 30.6 18.5443.1 28.88 40.09 48.28 36 30.6 15.95 34.05 23.71 36.64 48.28 54 24.5715.95 30.6 25 25 33.19 72 20.26 9.91 34.05 29.74 30.6 42.67 90 22.4118.54 37.5 35.78 25.86 40.52 108 18.54 18.1 34.48 31.04 25 43.1 12621.98 18.1 31.47 26.72 26.72 37.5 144 20.26 8.62 28.02 31.9 30.6 44.4162 35.78 21.55 35.35 27.16 35.35 50.43 180 42.67 20.69 29.74 28.4546.98 59.91

TABLE III Comparison of correct classification rates (CCR) for WearingCoat scenario of CASIA B without partitioning. Mean + GR- Angle (φ) pHOGLXP LBP SLBP std Method 0 12.93 7.33 9.91 7.33 15.09 17.67 18 13.79 9.059.91 12.07 15.95 19.4 36 12.07 7.33 15.95 13.79 15.52 25 54 9.05 12.0718.1 11.64 14.66 23.28 72 9.48 12.07 16.38 12.07 13.79 37.5 90 8.19 6.4715.09 13.79 12.5 34.48 108 9.48 9.48 13.79 11.64 15.09 38.36 126 10.789.48 17.24 13.36 12.93 24.57 144 11.21 13.79 10.78 11.21 10.78 19.83 16213.79 7.33 10.78 12.07 7.76 14.66 180 12.93 6.9 11.21 7.33 10.35 17.67

In the Tables I, II, and III, as depicted above, the LBP method and SLBPmethod are described by X. Yang et al. in, “Gait recognition based ondynamic region analysis,” Signal Processing, vol. 88, no. 9, pp.2350-2356, 2008, and by W. Zhang et al. in, “Local Gabor binary patternhistogram sequence: a novel non-statistical model for facerepresentation and recognition,” in Tenth IEEE International Conferenceon Computer Vision (ICCV), vol. 1, October 2005, pp. 786-791 Vol. 1,which are both incorporated herein by reference in their entirety.

Additionally, in order to evaluate the effect of partitioning on therecognition rate, a group of experiments for the cases ofnon-overlapping partitioning and overlapping portioning techniques wereperformed for the above described three scenarios. The results for thenon-overlapping partitioning are depicted below in Tables IV, V, and VI,respectively. Furthermore, the results for the overlapping partitioningscenario are depicted below in Tables VII, VIII, and IX. The resultsdemonstrate an improvement in the performance of gait recognition, whenpartitioning is applied, with a slightly higher performance for the caseof overlapping partitioning.

TABLE IV comparison of correct classification rates (CCR) for NormalWalking of CASIA B with different non-overlapping partitioning. Angle 57 8 10 16 (φ) Holistic regions regions regions regions regions 0 95.2699.14 99.14 99.14 99.14 98.28 18 97.41 98.71 98.71 99.57 99.57 99.14 3691.38 97.41 97.41 97.85 97.85 96.12 54 92.24 98.71 97.41 97.85 97.4196.55 72 91.81 96.98 97.41 97.85 97.85 96.55 90 93.1 97.41 98.71 98.7199.14 97.85 108 93.97 98.71 98.71 99.14 99.14 97.41 126 91.81 99.1498.71 98.28 98.71 97.85 144 91.38 96.98 96.55 97.41 97.41 95.69 16292.24 98.28 97.41 97.41 97.41 96.12 180 95.26 97.41 96.98 98.71 98.2897.85

TABLE V comparison of correct classification rates (CCR) for CarryingBag of CASIA B with different non-overlapping partitioning. Angle 5 7 810 16 (φ) Holistic regions regions regions regions regions 0 64.66 68.169.83 68.1 69.83 65.09 18 48.28 56.47 58.62 54.74 52.16 47.85 36 48.2853.45 50.43 52.59 50.86 52.16 54 33.19 56.9 56.47 64.66 62.5 51.29 7242.67 56.9 46.12 54.74 46.98 41.38 90 40.52 55.6 46.98 47.85 41.38 37.5108 43.1 55.17 44.83 48.28 39.66 40.95 126 37.5 47.85 41.81 42.67 42.2443.1 144 44.4 56.47 40.95 47.41 40.95 46.55 162 50.43 60.35 56.9 57.7656.47 61.64 180 59.91 68.97 69.83 67.67 65.52 65.09

TABLE VI comparison of correct classification rates (CCR) for WearingCoat of CASIA B with different non-overlapping partitioning. Angle 5 7 810 16 (φ) Holistic regions regions regions regions regions 0 17.67 2537.07 39.66 43.1 34.91 18 19.4 20.26 29.31 39.66 43.97 31.04 36 25 27.1631.9 37.07 42.24 31.9 54 23.28 26.29 36.21 43.97 50.86 40.52 72 37.533.19 39.22 50 50.43 40.52 90 34.48 35.78 39.22 44.4 46.12 30.17 10838.36 35.35 42.67 49.14 46.12 31.9 126 24.57 26.29 33.62 42.67 45.6931.9 144 19.83 25 32.76 35.78 41.38 26.72 162 14.66 23.71 31.04 38.7939.66 30.17 180 17.67 28.45 36.21 43.1 50 33.19

TABLE VII comparison of correct classification rates (CCR) for NormalWalking of CASIA B with different overlapping partitioning. Angle 5 7 810 16 (φ) Holistic regions regions regions regions regions 0 95.26 98.2898.71 98.28 97.85 98.28 18 97.41 97.85 99.14 99.14 98.28 99.14 36 91.3896.98 96.98 96.98 96.55 96.12 54 92.24 97.41 97.41 97.41 96.98 96.98 7291.81 97.85 96.98 96.98 96.98 96.55 90 93.1 97.41 98.71 98.71 98.2898.28 108 93.97 98.28 98.28 98.71 98.28 97.85 126 91.81 98.28 98.2897.85 97.41 98.28 144 91.38 95.26 95.69 96.12 95.69 96.55 162 92.2494.83 95.26 96.55 95.69 96.55 180 95.26 96.98 96.98 97.41 97.41 97.41

TABLE VIII comparison of correct classification rates (CCR) for CarryingBag of CASIA B with different overlapping partitioning. Angle 5 7 8 1016 (φ) Holistic regions regions regions regions regions 0 64.66 66.3861.21 65.09 63.79 70.69 18 48.28 45.69 49.57 53.02 48.71 53.45 36 48.2853.02 48.28 50 46.55 51.29 54 33.19 51.72 56.04 58.62 55.17 53.02 7242.67 50 38.36 43.54 37.93 45.69 90 40.52 44.83 37.5 44.83 39.22 41.38108 43.1 48.28 37.5 40.52 37.07 41.38 126 37.5 42.24 39.22 43.1 43.143.97 144 44.4 46.98 40.95 43.1 40.09 48.71 162 50.43 58.62 52.59 58.1955.17 62.07 180 59.91 62.93 57.76 69.83 64.66 66.81

TABLE IX comparison of correct classification rates (CCR) for WearingCoat of CASIA B with different overlapping partitioning. Angle 5 7 8 1016 (φ) Holistic regions regions regions regions regions 0 17.67 20.69 2532.76 34.91 33.62 18 19.4 17.67 23.71 31.47 33.62 32.76 36 25 22.4127.59 32.76 36.21 32.33 54 23.28 25.86 32.33 39.66 46.55 38.79 72 37.528.89 33.19 41.38 40.52 39.22 90 34.48 28.45 31.47 38.79 41.81 32.76 10838.36 29.31 34.48 43.97 43.54 33.62 126 24.57 23.71 29.31 37.93 38.7931.47 144 19.83 23.28 26.72 31.04 33.19 29.31 162 14.66 21.12 29.7435.78 37.07 31.04 180 17.67 22.85 32.76 39.66 40.95 35.35

Accordingly, by one embodiment of the present disclosure, the GEIsimages of CASIA B dataset are convolved with a Gabor filter bank toobtain the Gabor-based gait representation. Further, these Gaborpatterns are partitioned into non-overlapping and overlapping regions inorder to enhance and preserve as much discriminative gait information aspossible. Finally, several statistical features are investigated whichinclude at least the mean and standard deviation. Results illustrated inTables I-IX above indicate that the GR method of gait recognition asdescribed herein outperforms several other methods. Moreover, theresults demonstrate that using overlapping partitioning can furtherenhance the performance of gait recognition. Furthermore, thehighlighted cells in the above Tables I-IX depict the enhancedperformance of the gait recognition method of the present disclosure.

By one embodiment, several experiments are conducted on another dataset,OU-ISR-a Gait Database, and the performance of the gait recognitiontechnique described herein is compared against five additional methodsfor gait feature extraction. The dataset is a collection of gait videosof 34 persons walking on a treadmill from side view with of ninedifferent speeds from 2 km/h to 10 km/h at 1 km/h interval. Each personhas two sequences: one for “gallery” and one for “probe” at each speed.

The OU-ISIR is maintained by the Institute of Scientific and IndustrialResearch (ISIR), Osaka University (OU) “H. Iwama, M. Okumura, Y.Makihara and Y. Yagi, “The OU-ISIR Gait Database Comprising the LargePopulation Dataset and Performance Evaluation of Gait Recognition,” IEEETransactions on Information Forensics and Security, vol. 7, no. 5, pp.1511-1521, October 2012.”, and incorporated herein by reference in itsentirety.

In what follows, Tables X-XIII illustrate the performance of the gaitrecognition technique describe herein. Results demonstrate that thestatistical approach described herein outperforms all other methodsunder most speeds and uses four performance measures. To evaluate theeffect of partitioning on the performance, a group of experiments isdesigned. By one embodiment, correct classification rate (CCR)performance measure is used to test the level of improvement that can beobtained using partitioning. For non-overlapping partitioning, theresults are shown in Table XIV. Similarly, for overlapping partitioning,the results are shown in Tables XV. The results demonstrate improvementin the performance when partitioning is applied, with an edge foroverlapping.

TABLE X Comparison of correct classification rates (CCR) under differentwalking speeds using OU-ISIR-A database without partitioning. SpeedGEI + Mean + (km/h) pHOG LXP LBP SLBP GEI PCA LGXP LGBP SLGBP stdproposed 2 91.17 67.64 73.52 67.64 94.11 94.11 88.23 91.17 91.17 91.1794.12 3 88.23 55.88 67.64 61.76 91.17 94.11 85.29 91.17 88.23 88.23 1004 82.35 52.94 64.7 55.88 97.05 94.11 79.41 82.35 88.23 91.17 94.11 576.47 38.23 64.7 47.05 85.29 94.11 88.23 88.23 91.17 88.23 94.11 6 82.3547.05 58.82 55.88 88.23 88.23 85.29 97.05 94.11 91.17 97.05 7 91.1732.35 58.82 44.11 85.29 88.23 82.35 79.41 85.29 91.17 94.11 8 91.1747.05 61.76 67.64 85.29 91.17 82.35 88.23 91.17 85.29 91.17 9 94.1147.05 61.76 55.88 79.41 82.35 79.41 88.23 82.35 76.47 82.35 10 91.17 5079.41 64.7 91.17 91.17 82.35 91.17 82.35 82.35 91.17

TABLE XI Comparison of precisions under different walking speeds usingOU-ISIR-A database without partitioning. Speed GEI + Mean + (km/h) pHOGLXP LBP SLBP GEI PCA LGXP LGBP SLGBP std proposed 2 0.86 0.59 0.64 0.530.91 0.91 0.82 0.86 0.87 0.87 0.91 3 0.82 0.45 0.54 0.52 0.88 0.91 0.780.89 0.83 0.82 1 4 0.75 0.43 0.53 0.49 0.95 0.91 0.7 0.74 0.83 0.88 0.915 0.69 0.26 0.51 0.35 0.8 0.91 0.82 0.82 0.86 0.83 0.91 6 0.75 0.36 0.470.45 0.83 0.82 0.8 0.95 0.92 0.87 0.95 7 0.87 0.25 0.48 0.36 0.77 0.820.73 0.74 0.77 0.86 0.91 8 0.87 0.37 0.47 0.57 0.79 0.86 0.79 0.83 0.860.8 0.86 9 0.91 0.36 0.51 0.45 0.72 0.76 0.74 0.84 0.77 0.67 0.73 100.86 0.4 0.7 0.52 0.86 0.89 0.75 0.87 0.74 0.76 0.86

TABLE XII Comparison of Recalls under different walking speeds usingOU-ISIR-A database without partitioning. Speed GEI + Mean + (km/h) pHOGLXP LBP SLBP GEI PCA LGXP LGBP SLGBP std proposed 2 0.91 0.67 0.73 0.670.94 0.94 0.88 0.91 0.91 0.91 0.94 3 0.88 0.55 0.67 0.61 0.91 0.94 0.850.91 0.88 0.88 1 4 0.82 0.52 0.64 0.55 0.97 0.94 0.79 0.82 0.88 0.910.94 5 0.76 0.38 0.64 0.47 0.85 0.94 0.88 0.88 0.91 0.88 0.94 6 0.820.47 0.58 0.55 0.88 0.88 0.85 0.97 0.94 0.91 0.97 7 0.91 0.32 0.58 0.440.85 0.88 0.82 0.79 0.85 0.91 0.94 8 0.91 0.47 0.61 0.67 0.85 0.91 0.820.88 0.91 0.85 0.91 9 0.94 0.47 0.61 0.55 0.79 0.82 0.79 0.88 0.82 0.760.82 10 0.91 0.5 0.79 0.64 0.91 0.91 0.82 0.91 0.82 0.82 0.91

TABLE XIII Comparison of F1 measure under different walking speeds usingOU-ISIR-A database without partitioning. Speed GEI GEI + Mean + (km/h)pHOG LXP LBP SLBP [6] PCA LGXP LGBP SLGBP std proposed 2 0.88 0.62 0.660.58 0.92 0.92 0.84 0.88 0.88 0.88 0.92 3 0.84 0.48 0.58 0.55 0.89 0.920.8 0.9 0.84 0.84 1 4 0.77 0.45 0.57 0.51 0.96 0.92 0.73 0.77 0.85 0.890.92 5 0.71 0.29 0.55 0.38 0.81 0.92 0.84 0.84 0.88 0.85 0.92 6 0.770.39 0.5 0.48 0.85 0.84 0.81 0.96 0.92 0.88 0.96 7 0.88 0.27 0.51 0.370.8 0.84 0.76 0.76 0.8 0.88 0.92 8 0.88 0.4 0.51 0.6 0.81 0.88 0.8 0.840.88 0.82 0.88 9 0.92 0.39 0.54 0.48 0.74 0.78 0.75 0.85 0.78 0.7 0.7610 0.88 0.43 0.73 0.56 0.88 0.89 0.77 0.88 0.77 0.78 0.88

TABLE XIV Comparison of correct classification rates (CCR) underdifferent walking speeds using OU-ISIR-A database with differentnon-overlapping partitioning. Speed 5 7 8 10 16 (km/h) Holistic regionsregions regions regions regions 2 94.11 97.05 97.05 94.11 94.11 97.05 3100 100 100 100 100 100 4 94.11 100 97.05 100 97.05 100 5 94.11 94.1197.05 100 100 97.05 6 97.05 97.05 100 100 100 97.05 7 94.11 97.05 97.0597.05 97.05 97.05 8 91.17 97.05 97.05 10 100 97.05 9 82.35 97.05 94.1194.11 91.17 97.05 10 91.17 94.11 97.05 94.11 91.17 100

TABLE XV Comparison of correct classification rates (CCR) underdifferent walking speeds using OU-ISIR-A database with differentoverlapping partitioning. Speed 5 7 8 10 16 (km/h) Holistic regionsregions regions regions regions 2 94.11 97.05 97.05 91.17 91.17 97.05 3100 100 100 97.5 97.05 97.05 4 94.11 100 100 97.05 100 97.05 5 94.1191.17 97.05 100 100 97.05 6 97.05 97.05 100 100 100 97.05 7 94.11 97.0597.05 97.05 97.05 97.05 8 91.17 100 100 97.05 100 97.05 9 82.35 94.1191.17 91.17 91.17 97.05 10 91.17 97.05 94.11 94.11 91.17 100By one embodiment, in order to evaluate the effect of speed variationson the performance of the gait recognition approach described herein, agroup of experiments is designed as follows:

-   -   1. Setup A: All gallery and probe sets for all subjects under        all speeds are merged into one training dataset except the speed        to be tested. The excluded probe set is then used to evaluate        the performance.    -   2. Setup B: Only gallery sets for all subjects under all speeds        are merged and used to train the model. Then, probe sets of        different speeds are used to evaluate the performance.    -   3. Setup C: All gallery and probe sets for all subjects under        all speeds are merged into one training dataset. The performance        is evaluated using 5-fold cross validation.        The results for Setup A and B are shown in Table XVI, whereas        the per-class performance results for Setup C are shown in Table        XVII.

TABLE XVI Performance evaluation of the proposed method against walkingspeed variations of the subjects under surveillance using OU-ISIR-Adatabase. Training Setup Speed (km/h) Setup A Setup B 2 100 100 3 100100 4 100 100 5 100 100 6 97.05 97.05 7 94.11 94.11 8 100 97.05 9 91.1794.11 10 91.17 88.23

TABLE XVII 5-fold cross validation of all subjects under surveillanceusing OU-ISIR-A database. Subject_ID TP FP Precision Recall F-Measure a1 0 1 1 1 b 0.944 0 1 0.944 0.971 c 0.889 0.007 0.8 0.889 0.842 d 0.9440 1 0.944 0.971 e 0.944 0.003 0.895 0.944 0.919 f 1 0 1 1 1 g 0.9440.002 0.944 0.944 0.944 h 1 0 1 1 1 i 0.833 0.005 0.833 0.833 0.833 j0.889 0.003 0.889 0.889 0.889 k 1 0 1 1 1 l 1 0 1 1 1 m 0.944 0.0030.895 0.944 0.919 n 0.944 0.002 0.944 0.944 0.944 o 0.778 0.003 0.8750.778 0.824 p 0.889 0.005 0.842 0.889 0.865 q 0.944 0 1 0.944 0.971 r 10.003 0.9 1 0.947 s 1 0 1 1 1 t 0.944 0.003 0.895 0.944 0.919 u 0.889 01 0.889 0.941 v 1 0 1 1 1 w 0.944 0.008 0.773 0.944 0.85 x 0.944 0 10.944 0.971 y 0.944 0 1 0.944 0.971 z 1 0.002 0.947 1 0.973 aa 0.889 0 10.889 0.941 bb 1 0.003 0.9 1 0.947 cc 0.778 0 1 0.778 0.875 dd 0.9440.002 0.944 0.944 0.944 ee 0.889 0.007 0.8 0.889 0.842 ff 0.889 0 10.889 0.941 gg 0.944 0.002 0.944 0.944 0.944 hh 1 0 1 1 1 Overall 0.9380.002 0.942 0.938 0.938

Each of the functions of the above described embodiments may beimplemented by an image processing device such as a gait recognizingdevice that includes circuitry which is configured to perform thefunctions recited herein.

Furthermore, each of the functions of the described embodiments may beimplemented by one or more processing circuits. A processing circuitincludes a programmed processor (for example, processor 303 in FIG. 3),as a processor includes circuitry. A processing circuit also includesdevices such as an application-specific integrated circuit (ASIC) andcircuit components arranged to perform the recited functions. Thevarious features discussed above may be implemented by a computer system(or programmable logic). FIG. 3 illustrates such a computer system 301.

The computer system 301 includes a disk controller 306 coupled to thebus 302 to control one or more storage devices for storing informationand instructions, such as a magnetic hard disk 307, and a removablemedia drive 308 (e.g., floppy disk drive, read-only compact disc drive,read/write compact disc drive, compact disc jukebox, tape drive, andremovable magneto-optical drive). The storage devices may be added tothe computer system 301 using an appropriate device interface (e.g.,small computer system interface (SCSI), integrated device electronics(IDE), enhanced-IDE (E-IDE), direct memory access (DMA), or ultra-DMA).

The computer system 301 may also include special purpose logic devices(e.g., application specific integrated circuits (ASICs)) or configurablelogic devices (e.g., simple programmable logic devices (SPLDs), complexprogrammable logic devices (CPLDs), and field programmable gate arrays(FPGAs)).

The computer system 301 may also include a display controller 309coupled to the bus 302 to control a display 310, for displayinginformation to a computer user. The computer system includes inputdevices, such as a keyboard 311 and a pointing device 312, forinteracting with a computer user and providing information to theprocessor 303. The pointing device 312, for example, may be a mouse, atrackball, a finger for a touch screen sensor, or a pointing stick forcommunicating direction information and command selections to theprocessor 303 and for controlling cursor movement on the display 310.

The processor 303 executes one or more sequences of one or moreinstructions contained in a memory, such as the main memory 304. Suchinstructions may be read into the main memory 304 from another computerreadable medium, such as a hard disk 307 or a removable media drive 308.One or more processors in a multi-processing arrangement may also beemployed to execute the sequences of instructions contained in mainmemory 304. In alternative embodiments, hard-wired circuitry may be usedin place of or in combination with software instructions. Thus,embodiments are not limited to any specific combination of hardwarecircuitry and software.

As stated above, the computer system 301 includes at least one computerreadable medium or memory for holding instructions programmed accordingto any of the teachings of the present disclosure and for containingdata structures, tables, records, or other data described herein.Examples of computer readable media are compact discs, hard disks,floppy disks, tape, magneto-optical disks, PROMs (EPROM, EEPROM, flashEPROM), DRAM, SRAM, SDRAM, or any other magnetic medium, compact discs(e.g., CD-ROM), or any other optical medium, punch cards, paper tape, orother physical medium with patterns of holes.

Stored on any one or on a combination of computer readable media, thepresent disclosure includes software for controlling the computer system301, for driving a device or devices for implementing the invention, andfor enabling the computer system 301 to interact with a human user. Suchsoftware may include, but is not limited to, device drivers operatingsystems, and applications software. Such computer readable media furtherincludes the computer program product of the present disclosure forperforming all or a portion (if processing is distributed) of theprocessing performed in implementing any portion of the invention.

The computer code devices of the present embodiments may be aninterpretable or executable code mechanism, including but not limited toscripts, interpretable programs, dynamic link libraries (DLLs), Javaclasses, and complete executable programs. Moreover, parts of theprocessing of the present embodiments may be distributed for betterperformance, reliability, and/or cost.

The term “computer readable medium” as used herein refers to anynon-transitory medium that participates in providing instructions to theprocessor 303 for execution. A computer readable medium may take manyforms, including but not limited to, non-volatile media or volatilemedia. Non-volatile media includes, for example, optical, magneticdisks, and magneto-optical disks, such as the hard disk 307 or theremovable media drive 308. Volatile media includes dynamic memory, suchas the main memory 304. Transmission media, on the contrary, includescoaxial cables, copper wire and fiber optics, including the wires thatmake up the bus 302. Transmission media also may also take the form ofacoustic or light waves, such as those generated during radio wave andinfrared data communications.

Various forms of computer readable media may be involved in carrying outone or more sequences of one or more instructions to processor 303 forexecution. For example, the instructions may initially be carried on amagnetic disk of a remote computer. The remote computer can load theinstructions for implementing all or a portion of the present disclosureremotely into a dynamic memory and send the instructions over atelephone line using a modem. A modem local to the computer system 301may receive the data on the telephone line and place the data on the bus302. The bus 302 carries the data to the main memory 304, from which theprocessor 303 retrieves and executes the instructions. The instructionsreceived by the main memory 304 may optionally be stored on storagedevice 307 or 308 either before or after execution by processor 303.

The computer system 301 also includes a communication interface 313coupled to the bus 302. The communication interface 313 provides atwo-way data communication coupling to a network link 314 that isconnected to, for example, a local area network (LAN) 315, or to anothercommunications network 316 such as the Internet. For example, thecommunication interface 313 may be a network interface card to attach toany packet switched LAN. As another example, the communication interface313 may be an integrated services digital network (ISDN) card. Wirelesslinks may also be implemented. In any such implementation, thecommunication interface 313 sends and receives electrical,electromagnetic or optical signals that carry digital data streamsrepresenting various types of information.

The network link 314 typically provides data communication through oneor more networks to other data devices. For example, the network link314 may provide a connection to another computer through a local network315 (e.g., a LAN) or through equipment operated by a service provider,which provides communication services through a communications network316. The local network 314 and the communications network 316 use, forexample, electrical, electromagnetic, or optical signals that carrydigital data streams, and the associated physical layer (e.g., CAT 5cable, coaxial cable, optical fiber, etc.). The signals through thevarious networks and the signals on the network link 314 and through thecommunication interface 313, which carry the digital data to and fromthe computer system 301 may be implemented in baseband signals, orcarrier wave based signals.

The baseband signals convey the digital data as unmodulated electricalpulses that are descriptive of a stream of digital data bits, where theterm “bits” is to be construed broadly to mean symbol, where each symbolconveys at least one or more information bits. The digital data may alsobe used to modulate a carrier wave, such as with amplitude, phase and/orfrequency shift keyed signals that are propagated over a conductivemedia, or transmitted as electromagnetic waves through a propagationmedium. Thus, the digital data may be sent as unmodulated baseband datathrough a “wired” communication channel and/or sent within apredetermined frequency band, different than baseband, by modulating acarrier wave. The computer system 301 can transmit and receive data,including program code, through the network(s) 315 and 316, the networklink 314 and the communication interface 313. Moreover, the network link314 may provide a connection through a LAN 315 to a mobile device 317such as a personal digital assistant (PDA) laptop computer, or cellulartelephone.

While aspects of the present disclosure have been described inconjunction with the specific embodiments thereof that are proposed asexamples, alternatives, modifications, and variations to the examplesmay be made. It should be noted that, as used in the specification andthe appended claims, the singular forms “a,” “an,” and “the” includeplural referents unless the context clearly dictates otherwise.

1. A method of identifying a human subject by gait recognition, themethod comprising: obtaining a number of image frames of the humansubject showing the human subject's walking style and storing the imageframes in a memory, receiving by circuitry a gait sequence, the gaitsequence including a predetermined number of the image frames of thehuman subject; processing by circuitry, the received gait sequence togenerate a gait-energy-image (GEI), wherein the GEI is a sum of walkingsilhouette images divided by the number of images; computing bycircuitry, a plurality of Gabor filter responses by convoluting thegenerated GEI with a bank of Gabor filters, the filter bank including afirst predetermined number of unique scales, and a second predeterminednumber of unique orientations, partitioning by circuitry, each Gaborfilter response of the computed plurality of Gabor filter responses,into a predetermined number of overlapping regions, wherein thepartitioning is performed as a fraction of the height of the humansubject and width of the human subject, and is denoted by horizontal andvertical lines; extracting by circuitry, a predetermined number ofstatistical features only from the overlapping regions, the extractedstatistical features corresponding to texture content of the subject;and identifying by circuitry, the human subject based on aclassification of the extracted statistical features.
 2. The method ofclaim 1, wherein the first predetermined number of scales is five andthe second predetermined number of orientations is eight.
 3. The methodof claim 2, wherein a magnitude of the plurality of Gabor filterresponses is equal to a scalar multiplication of the first predeterminednumber and second predetermined number.
 4. The method of claim 1,wherein each overlapping region of the predetermined number ofoverlapping regions has a unique size.
 5. The method of claim 1, whereinthe predetermined number of extracted statistical features is eight, theextracted statistical features including a mean, a standard deviation,energy content, correlation, contrast, entropy, homogeneity, andsmoothness.
 6. The method of claim 1, further comprising: merging bycircuitry, the predetermined number of extracted statistical features toform a global feature vector.
 7. The method of claim 1, wherein theclassification is performed based on a SVM classifier. 8-20. (canceled)